The present invention relates to the field of video image processing, and in particular to motion-vector-aided pixel interpolation.
Motion or displacement vectors (hereinafter “motion vectors”) in image processing define a position change of an object or of an image region from a first position in a first image to a second position in a second image. The two images can be acquired in temporal succession and thus visualize a motion of the object or image region over time. The motion vector in such cases is used, for example, in the interpolation of one or more intermediate images that reflect the position and/or other characteristics of the object/image region at one or more times between the acquisition of the first and second images, for example in 50 Hz/100 Hz image conversion or in the generation of “artificial slow motion.” The two images can also be acquired simultaneously from first and second camera positions. In this case the motion vector is used to determine an intermediate image that shows the object/image region from a virtual third camera position lying between the first and second camera positions.
Techniques for estimating motion vectors for selected image regions, which in the extreme case comprise one pixel, on the basis of first and second images are well known. Typically, the image region at the initial point of the motion vector in the first image is determined along with the image region at the terminal point of the motion vector in the second image. An image region of the intermediate image can then be interpolated by using either the image region at the initial point of the motion vector in the first image or the image region at the terminal point of the motion vector in the second image. The position of the interpolated image region depends on the temporal position of the intermediate image between the two original images or the spatial position of the virtual camera position relative to the first and second camera positions.
Determination of the motion vector, however, usually involves errors, so that in known techniques the item of video information of a pixel of the intermediate image is determined by mixing together the item of video information of the pixel at the initial point of the motion vector in the first image and the item of video information of the pixel at the terminal point of the motion vector in the second image. For example, the items of video information of the two pixels can be weighted for mixing, depending on the application.
Techniques are also known in which a plurality of motion vectors are determined in each case for the individual image regions. Correspondingly, a plurality of motion vectors are likewise associated with a pixel in the intermediate image. In one known technique, the video information value of the pixel of the intermediate image is interpolated by applying median filtering to the video information values of the initial points of the motion vectors in the first image and the video information values of the terminal points of the motion vectors in the second image. Such a technique is described for example in U.S. Pat. No. 5,534,946.
What is needed is an easily implemented method and apparatus for motion-vector-aided pixel interpolation.